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ABSTRACT 

Primary validation of the Nor Cal questionnaire was 
accomplished in Phase 2 of the Nor Cal Attrition Study. The results 
of the primary validation were reported in the document entitled, 
"Phase 2 Final Report," (ED 039 879) . The primary validation showed 
that the consortium-wide empirical validity varied from .65 to .67 
depending upon whether or not the ACT aptitud.e test scores were 
incorporated into the predictive equation. In terms of the primary 
validation the research question was, "What proportion of first-time, 
full-time freshmen were correctly identified by the Nor Cal 
questionnaire predictive equation as either potential dropouts or 
potential persisters?" For the secondary evaluation, another research 
question also dealing with the predictive validity of the instrument 
emerged: "Are there significant differences between the subsequent 
performance levels of these groups?" This secondary validation is 
therefore concerned with comparing attrition rates, units completed, 
and grade point averages betw^n groups of entering freshmen for whom 
appropriate scores were known. The results of this validation study 
show that when performance levels of students who are designated as 
potential dropouts by the Nor Cal questionnaire are compared with the 
levels of other students, it is clear that potential dropouts do: (1) 
have a signifies “tly higher dropout rate, (2) complete fewer units, 
and (3) have lower grades. (Author/AL) 
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FURTHER VALIDATION OF THE NOR CAL QUESTIONNAIRE 
SECONDARY VALIDATION 1 



Primary validation of the Nor Cal Questionnaire was accomplished in Phase 
2 of the Nor Cal Attrition Study. The results of the primary validation were re- 
ported in the document entitled, "Phase 2 Final Report," (MacMillan, 1970). The 
primary validation showed that the consortium-wide empirical validity varied from 
.65 to .67 depending upon whether or not the ACT aptitude test scores were incor- 
porated into the predictive equation. In terms of the primary validation the re- 
search question was, "What proportion of first -time, full-time freshmen were 
correctly identified by the Nor Cal questionnaire predictive equation as either po- 
tential dropouts or potential persisters?" In this validation study the operational 
definitions were as follows: "Given that the Sum-1 scores range from +45 to -45, 
a ’potential dropout’ is one who has a Sum-1 discriminant score of above 0, a 
'potential persister' is one who has a Sum-1 discriminant score of below 0." Thus 
a fairly high level of predictive validity was attained even when a "cutting score" 
of zero (0) was used. 

As Phase 3 was being planned in detail, another research question emerged. 
The second question also dealt with the predictive validity of the instrument. Given 
initial categorization of first-time, full-time freshmen on the basis of their Nor Cal 



Primary validation was done with a data base that included twenty-two 
Northern California Community Colleges and 16,488 students. By contrast this 
secondary validation was done with a data base that included five colleges and 
5612 students. 
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Sum-1 scores, the second research question was: "Are there significant differ- 
ences between the subsequent performance levels of these groups?" Using a Sum- 

2 

1 score of +10 or higher to designate a potential dropout the question became: 

"Do students who have Sum-1 scores of +10 or higher differ in their performance, 
levels from other students ?" At first the question c entered on the differences in 
attrition rates between identified potential dropouts and other groups. But when 
five college representatives indicated their willingness to investigate units com- 
pleted and grade point average in addition to attrition, the research question was 
broadened to include comparisons using all three criteria. 

* The secondary validation would, therefore, be concerned with comparing 
attrition rates, units completed, and grade point averages between groups of 
entering freshmen for whom Sum-1 scores were known. These comparisons 
would be made between potential dropouts, those having Sum-1 scores of +10 or 
higher, and other groups. 

Of half-a-dozen nearby colleges that were contacted, five responded in the 
affirmative; five would gather the additional data necessary to do the secondary 
validation. The Nor Cal college representative at each of these five community 
colleges must be commended for successfully directing the data collection. Each 
representative collected data on attrition rate and units completed for their col- 
lege's Fall 1969 Nor Cal sample. Where possible data from more than the fall 
quarter or semester was obtained. Also where possible data on grade point 
averages were also collected. The following people deserve credit for seeing 



The "plus -ten or higher" Sum-1 score range was recommended by Dr. 
MacMillan for use as the operational definition of a potential dropout. (MacMillan, 
1970). 
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that this data was collected and made available for statistical analysis: Dr. Donald 
Denevi, Merritt College; Dr. Paul Preising, San Jose City College; Mr. David 
Shaw, De Anza College; Mr. Irel Lowe, Foothill College; and Dr. William Wenrich 
College of San Mateo. Needless to say without the efforts of these representatives 
the secondary validation which follows could not have been done. 

While the representatives began collecting the additional data, a start was 
made in the direction of the secondary validation study. An attrition rate compari- 
son could be made by abstracting data from the Phase 2 Final Report. It should be 
kept in mind that zero (0) was used as the cutting score. Table 1 shows the first 
step in gaining the necessary information. This table shows the determination of 
the attrition rates at each college and for the consortium as a whole. 



Table 1 on Page 4 



The second step in gaining the required information came from the results 
of Table 2. Table 2 shows the attrition rate of the group of students who had Sum- 
1 scores at or above zero (0). 



Table 2 on Page 5 



After these calculations were accomplished, it was simple to compare the 
attrition rate of students who had positive Sum-1 scores against those who had 
negative Sum-1 scores. A statistically significant difference was found as is 



shown in Table 3. 



Table 1 



Attrition Rate by College and for the Consortium as a Whole 

Data from Fall, 1969 



College 


Number of Students 
That Actually Withdrew 
(WD-yes, Pius WD-No) 


Total Number 
of Students 


Attrition Rate 


A 


71 


470 


.1510 


B 


40 


442 


.0904 


C 


24 


900 


.0266 


D 


174 


1787 


.0973 


E 


34 


305 


.1114 


F 


94 


977 


. 0962 


G 


78 


1640 


.0475 


H 


46 


295 


.1559 


I 


1 


289 


.0034 


J 


15 


229 


.0655 


K 


38 


786 


. 0483 


L 


62 


1005 


.0616 


M 


73 


997 


.0732 


N 


47 


300 


.1566 


0 


64 


627 


.1020 


P 


24 


377 


.0636 


Q 


129 


1060 


.1216 


R 


28 


150 


.1866 


S 


158 


1006 


.1570 


T 


7 


131 


.0534 


U 


57 


2248 


. 0253 


V 


60 


467 


.1284 


TOTALS 


1324 


16488 


.0803 



Table 2 



Attrition Rate of Students Having Sum-1 Scores 
That Were Above Zero (0) 



By College and for the Consortium as a Whole 



College 


Number of Students 
Who Were Predicted to Withdraw 
(WD-Yes Plus Pers-No) 


Number Who Did Withdraw 


A 


199 


34 


B 


149 


19 


C 


298 


11 


D 


557 


79 


E 


93 


13 


F 


270 


27 


G 


532 


40 


H 


121 


17 


I 


62 


0 


J 


116 


13 


K 


217 


18 


L 


340 


26 


M 


319 


33 


N 


61 


16 


0 


240 


36 


P 


153 


14 


Q 


440 


55 


R 


59 


15 


S 


360 


78 


T 


38 


4 


U 


749 


27 


V 


229 


38 


TOTALS 


5602 


613 
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Table 3 



Nor Cal Consortium Level Validation Using Zero (0) 

As Cutting Score 



Withdrew 

Persisted 

Total 

Attrition Rate 



Sum-1 Scores 



Below 0 


Above 0 


711 


613 


10175 


4989 


10886 


5602 


6.53% 


10.94% 



Z = 9.87 

p < .0001 



Since using zero (0) as a cutting score had resulted in a significant differ- 
ence, it was hoped that a cutting score of plus ten (+10) would do the same for the 
smaller individual college samples. This hope was realized as Tables 4 through 
11* clearly indicate. 



Table 4 



Comparison of Attrition Rates for Secondary Validation Study 
Data From Foothill College, Fall Quarter, 1969 



Withdrew 

Persisted 

Total 

Attrition Rate 



Sum-1 Score 



-10 and Below 


+10 and Above 


25 


32 


185 


96 


210 


128 


11. 9% 


25. 0% 



Z = 3.12 

p < .001 



Table 5 



Comparison of Attrition Rates for Secondary Validation Study 
Data From Foothill College, Spring Quarter. 1970 





Sum-] 


Score 




-10 and Below 


+10 and Above 


Withdrew 


78 


60 


Persisted 


132 


68 


Total 


210 


128 


Attrition Rate 


37.1% 


46. 9% 



Z = 1.77 
p < .14 
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Table 6 



Comparison of Attrition Rates for Secondary Validation Study 
Data From De Anza College, Fall Quarter, 1969 



Withdrew 

Persisted 

Total 

Attrition Rate 



Sum-1 Score 



-10 and Below 


+10 and Above 


56 


69 


244 


201 


300 


270 


18.7% 


25.6% 



Z = 1.98 
p < .024 



Table 7 



Comparison of Attrition Rates for Secondary Validation Study 
Data From De Anza College. Spring Quarter, 1970 





Sum-] 


Score 




-10 and Below 


+10 and Above 


Withdrew 


102 


149 


Persisted 


198 


121 


Total 


300 • 


270 


Attrition Rate 


34.0% 


55.2% 



Z = 5.09 

p < .001 



Table 8 



Comparison of Attrition Rates for Secondary Validation Study 
Data from San Jose City College, Fall Semester. 1969 





Sum-] 


Score 




-10 and Below 


+10 and Above 


Withdrew 


149 


58 


Persisted 


569 


128 


Total 


718 


186 


Attrition Rate 


20.8% 


31.2% 



Z = 2.97 
p < .015 



o 
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Table 9 



Comparison of Attrition Rates f < > r Secondary Validation Study 
Data From San Jose City College for the Entire Academic 

School Year, 1969-70 
(Three Groups) 





-10 and Below 


Sum-1 Score 
Between 


+10 and Above 


Withdrew 


40 


166 


84 


Persisted 


130 


382 


102 


Total 


170 


548 


186 


Attrition Rate 


23.5% 


30.3% 


45.2% 



» u /O 

X*^21.12 

p < .001 



Table 10 



Comparison of Attrition Rates for Secondary Validation Study 
Data jVom San Jose City Co ll t ge for the Entire Academic 

School Yi ar. 1969-70 
(Two Groups) 



Withdrew 

Persisted 

Total 

Attrition Rate 



Sum-1 Score 



-10 and Below 


+10 and Above 


40 


84- 


130 


102 


170 


186 


23.5% 


45.2% 



Z = 4.28 

p< .001 



Table 11 

Comparison of Attrition Rates for Secondary Validation Study 
Data From College of San Mateo. Fall Semester. 1969 



Withdrew 

Persisted 

Total 

Attrition Rate 



Sum-1 Score 



-10 and Below 


+10 and Above 


17 


76 


400 


619 


417 


695 


4.1% 


10.9% 



Z =4.00 

p< .001 



o 

ERIC 

SiflraiMmTiaa 




Table 12 



Comparison of Attrition Rates for Secondary Validation Study 
Data From Merritt College, Fall Semester, 1969 





Sum-] 


Score 




-10 and Below 


+10 and Above 


Withdrew 


12 


13 


Persisted 


181 


80 


Total 


193 


93 


Attrition Rate 


6.2% 


14.0% 



Z = 2.18 
p < .015 



The Nor Cal questionnaire thus demonstrated high predictive validity. 

Seven of eight tests resulted in significance levels of .03 or better. It was clear 
that students having Sum-1 scores of plus -ten (+10) or greater did have high attri- 
tion rates . 

• i 

The next variable to be investigated was number of units completed in a 
given semester or quarter. For this investigation entering freshmen were divided 
into three groups based on their Sum-1 scores. One group was the potential drop- 
outs, those whose Sum-1 score was at or above "plus -ten" (+10). Another group 
included students whose Sum-1 score was at or below "minus -ten" (-10). The 
third group included those whose Sum-1 score was between "minus-ten" (-10) , 
and "plus-ten" (+10). Statistically significant differences exist between the three 
groups of students on the variable of number of units completed, as Tables 13 



through 36 show. 



Table 13 



Descriptive Statistics for Validation Study of Data 
From Foothill College, Fall Quarter, 1969 

Variable is Units Completed 



Group -10 and Below Between . +10 and Above 



Sample Size 


210 


487 


128 


Mean 


10.4857 


10.2115 


7.. 6992 


Standard Deviation 


5.4192 


5.6195 


6.1161 



Table 14 



Analysis of Variance for Validation Study of Data 
From Foothill College, Fall Quarter. 1969 

Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


739.1975 


2 


369.5986 


11. 580 


Within Groups 


26235.4492 


822 


31.9166 




Total 


26974.6445 


824 








P< 


.0005 







Table 15 



Descriptive Statistics for Validation Study of Data 
From Foothill College, Winter Quarter, 1970 

Variable is Units Completed 



Group 


-10 and Below 


Between 


+10 and Above 


Sample Size 


210 


487 


128 


Mean 


8.8024 


8.8788 


6.5352 


Standard Deviation 


6.4635 


6.7185 


6.0484 



Table 16 



Analysis of Variance for Validation Study of Data 
From Foothill College, Winter Quarter, 1970 



Variable is Units Completed 



Sum of Squares DF Mean Square F Ratio 



Between Groups 


583.2417 


2 


291.6208 . 


6.7879 


Within Groups 


35314.7070 


822 


42.9619- 


J 



H 



Total 


35897.9453 


824 




P< 


.005 



Table 17 



Descriptive Statistics for Validation Study of Data 
From Foothill College, Spring Quarter, 1970 

Variable is Units Completed 



Group 


-10 and Below 


Between 


+10 and Above 


Sample Size 


210 


487 


128 


Mean 


7.4667 


7.0708 


5.0000 


Standard Deviation 


6.8856 


6.7427 


5.9631 



Table 18 

Analysis of Variance for Validation Study of Data 
From Foothill College, Spring Quarter, 1970 

Variable is Units Completed 





Sum of Sauares 


DF 


Mean Square 


F Ratio 


Between Groups 


541.6895 


2 


270. 8447 


6.0961 


Within Groups 


36520.7344 


822 


44.4291 




Total 


37062.4219 


824 







p < .005 



Table 19 



Descriptive Statistics for Validation Study of Data 
From Foothill College for the Entire Academic 
School Year. 1969-70 

Variable is Units Completed 



Group 


-10 and Below 


Between 


+10 and Above 


Sample Size 


210 


487 


128 


Mean 


26.7548 


26.1807 


19.2344 


Standard Deviation 


15.9702 


16.6055 


16.2306 



Table 20 



Analysis of Variance for Validation Study of Data 
From Foothill College for the Entire Academic 
School Year. 1969-70 

Variable is Units Completed 

Sum of Squares DF Mean Square F Ratio 

Between Groups 5529.3516 2 2764.6758 10.2938 



Table 21 



Descriptive Statistics for Validation Study of Data 
From De Anza College . Fall, 1969 

Variable is Units Completed 



Group -10 and Below Between +10 and Above 



Sample Size 


300 


705 


270 


Mean 


10.2867 


9.4355 


7.8611 


Standard Deviation 


6.0382 


6.0332 


6.0114 



Table 22 

Analysis of Variance for Validation Study of Data 
From De Anza College, Fall, 1969 

Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


863.9924 


2 


431.9961 


11. 8818 


Within Groups 


46247.2930 


1272 


36.3579 




Total 


47111.2852 


1274 







p< .0005 



Table 23 

Descriptive Statistics for Validation Study of Data 
From De Anza College. Winter. 1970 

Variable is Units Completed 



Group 


-10 and Below 


Between 


+10 anchAbove" 


Sample Size 


300 


705 


270 


Mean 


9. 1250 


8.0567 


5.8889 


Standard Deviation 


6.5269 


6.7855 


6.4272 



Table 24 

Analysis of Variance for Validation Study of Data 
From De Anza College. Winter. 1970 

Variable is Units Completed 



Sum of Squares DF Mean Square F Ratio 



Between Groups 


1556.2268 


2 


778.1133 


17.5912 


Within Groups 


56264.5234 


1272 


44.2331 




Total 


57820.7500 


1274 


• 





p< .0005 
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Table 25 



Descriptive Statistics for Validation Study of Data 
From De Anza College. Spring, 1970 

Variable is Units Completed 



Group 


-10 and Below 


Between 


+10 and Above 


Sample Size 


300 


705 


270 


Mean 


8.2333 


6. 8433 


4. 8333 


Standard Deviation 


6.7751 


6.8277 


6.1652 



Table 26 

Analysis of Variance for Validation Study of Data 
From De Anza College, Spring, 1970 

Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


1658.0508 


2 


829.0254 


18.5758 


Within Groups 


56768.5234 


1272 


44.6293 




Total 


58426.5742 


1274 







p < .0005 



Table 27 



Descriptive Statistics for Validation Study of Data 
From De Anza College for the Entire Academic 
School Year. 1969-70 
Variable is Units Completed 



Group -10 and Below Between +10 ana Above 



Sample Size 


300 


705 


270 


Mean 


27.6450 


24.3354 


18. 5833 


Standard Deviation 


17.4912 


17.6092 


16.4752 



Table 28 

Analysis of Variance for Validation Study of Data 
From De Anza C olle ge for the Entire Academic 
School Y ear. 1969-70 
Variable is Units Completed 

Sum of Squares DF Mean Square F Ratio 



Between Groups 


11973.2695 


2 


5986.6328 


19. 8933 


Within Groups 


382791.6875 


1272 


300. 9368 


* 


Total 


394764. 9375 


1274 








p < .0005 
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Table 29 



Descriptive Statistics for Validation Study of Data 
From San Jose City College, Fall, 1969 

Vari p ble is Units Completed 



Group -10 and Below Between +10 and Above 



Sample Size 


170 


548 


186 


Mean 


9.1382 


8.1679 


5.7984 


Standard Deviation 


5.3557 


5.9188 


5.3226 



Table 30 

Analysis of Variance for Validation Study of Data 
From San Jose City College, Fall, 1969 

Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


1120.2451 


2 


560.1226 


17.2530 


Within Groups 


29251.2266 


901 


32.4653 


• 


Total 


30371.4687 


903 







p< .0005 



Table 31 

Descriptive Statistics for Validation Study of Data 
From San Jose City College for the Entire Academic 
School Year, 1969-70 

Variable is Units Completed 



Group -10 and Below Between +10 and Above 



Sample Size 


170 


047 


185 


Mean 


16.4441 


14.0722 


9.1919 


Standard Deviation 


11.3082 


11.7467 


10.0881 




Table 32 

Analysis of Variance for Validation Study of Data 
From San Jose City College for the Entire Academic 
School Year, 1969-70 



Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


5085.8945 


2 


2542.9473 


19.7631 


Within Groups 


115675.8750 


899 


128.6717 


J 


Total 


120761.7500 


901 







p< .0005 
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Table 33 



Descriptive Statistics for Validation Study of Data 
From San Jose City College for the Entire Academic 



Variable is Units Completed 



Group -10 and Below Between +10 and Above 



Sample Size 


130 


379 


103 


Mean 


21.4115 


20.1280 


16.0728 


Standard Deviation 


7.8234 


8.8036 


8.5035 



Table 34 

Analysis of Variance for Validation Study of Data 
From San Jose City College for the Entire Academic 



Variable is Units Completed 



Sum of Squares DF Mean Square F Ratio 



Between Groups 


1805.1292 


2 


902.5645 


12.33321 


Within Groups 


44567.6992 


609 


73.1818 




Total 


46372.8281 


611 








PC 


.0005 







Table 35 

Descriptive Statistics for Validation Study of Data 
From College of San Mateo, Fall, 1969 

Variable is Units Completed 



-10 and Below Between +10 and Above 



Sample Size 


417 


553 


695 


Mean 


13. 1511 


11.6094 


11.4095 


Standard Deviation 


10.1796 


8.7977 


9. 9796 



Table 36 

Analysis of Variance for Validation Study of Data 
From College of San Mateo . Fall, 1969 

Variable is Units Completed 





Sum of Squares 


DF 


Mean Square 


F Ratio 


Between Groups 


866.3604 


2 


433.1802 


4.6463 


Within Groups 


154949. 1875 


1662 


93.2305 




Total 


155815.5000 


1664 







p < .025 



Similarly, as Tables 37 and 38 illustrate, the potential dropouts as a group 



do attain lower grade point averages as well. 



Table 37 

Descriptive Statistics for Validation Study of Data 
From San Jose City College for the Entire Academic 
School Year. 1969-70 



Variable is Grade Point Average (Including Dropouts) 



Group 


-10 and Below 


Between 


+10 and Above 


Sample Size 


170 


547 


185 


Mean 


1.6842 


1. 5790 


1.1941 


Standard Deviation 


1.0288 


1.1546 


1. 1446 



Table 38 

Analysis of Variance for Validation Study of Data 
From San Jose City College for the Entire Academic 
School Year. 1969-70 

Variable is Grade Point Average (Including Dropouts) 



Sum of Squares DF Mean Square F Ratio 



Between Groups 


26.1361 


2 


13.0681 


10.2362 


Within Groups 


1147.7153 


899 


1.2767 




Total 


1173. 8513 


901 







p< .0005 



Summary 

Comparisons of attrition rates are summarized in Table 39. Those stu- 
dents who have Sum-1 score at or above plus ten (+10) and are therefore identified 
by the Nor Cal questionnaire as being potential dropouts do in fact drop out at rates 
statistically above those students in the other group. Seven out of eight analyses 
have significance levels beyond .03 and three have significant levels beyond the 
. 001 level. As the table shows, the students with Sum-1 scores of plus ten (+10) 
or higher are in fact potential dropouts. In fact these students drop out at a rate 



Table 39 
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Mean ratio of attrition rates for all comparisons shown above = 1. 87 
Ratio of attrition rate for the one entire academic year = 1. 92 
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Mean difference in units completed for Fall = 2.58 

Mean difference in units completed for academic year = 7.94 



almost double the others. 



Table 39 on Page 17 



Comparisons based on number of units completed in a given quarter or 
semester indicate the potential dropouts complete fewer units than do the other 
students. On the average if they do not entirely drop-out, they do drop about one 
course during their first semester or quarter they attend. Over the year the 
drop in units completed is greater, about eight units are dropped. It can be said 
that potential dropouts do complete significantly fewer units than do other students. 
Of eleven comparisons ten have significance levels of .005 or greater as shown in 
Table 40. 



Table 40 on Page 18 



Conclusion 

When performance levels of students who are designated as potential drop- 
outs by the Nor Cal questionnaire are compared with the levels of other students, 
it is clear that potential dropouts: do have a significantly higher dropout rate, do 
complete fewer units, and do have lower grades. Thus the Nor Cal questionnaire 



is shown to be valid. 
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